Accelerated variance-reduced methods for saddle-point problems
نویسندگان
چکیده
We consider composite minimax optimization problems where the goal is to find a saddle-point of large sum non-bilinear objective functions augmented by simple regularizers for primal and dual variables. For such problems, under average-smoothness assumption, we propose accelerated stochastic variance-reduced algorithms with optimal up logarithmic factors complexity bounds. In particular, strongly-convex-strongly-concave, convex-strongly-concave, convex-concave objectives. To best our knowledge, these are first nearly-optimal this setting. • First bounds empirical problems.
منابع مشابه
Stochastic Variance Reduction Methods for Saddle-Point Problems
We consider convex-concave saddle-point problems where the objective functionsmay be split in many components, and extend recent stochastic variance reductionmethods (such as SVRG or SAGA) to provide the first large-scale linearly conver-gent algorithms for this class of problems which are common in machine learning.While the algorithmic extension is straightforward, it come...
متن کاملCertified Reduced Basis Methods for Parametrized Saddle Point Problems
We present reduced basis approximations and associated rigorous a posteriori error bounds for parametrized saddle point problems. First, we develop new a posteriori error estimates that, unlike earlier approaches, provide upper bounds for the errors in the approximations of the primal variable and the Lagrange multiplier separately. The proposed method is an application of Brezzi’s theory for s...
متن کاملProbing Methods for Generalized Saddle-Point Problems
Several Schur complement-based preconditioners have been proposed for solving (generalized) saddlepoint problems. We consider probing-based methods for approximating those Schur complements in the preconditioners of the type proposed by [Murphy, Golub and Wathen ’00], [de Sturler and Liesen ’03] and [Siefert and de Sturler ’04]. This approach can be applied in similar preconditioners as well. W...
متن کاملProbing Methods for Saddle-point Problems
Abstract. Several Schur complement-based preconditioners have been proposed for solving (generalized) saddle-point problems. We consider matrices where the Schur complement has rapid decay over some graph known a priori. This occurs for many matrices arising from the discretization of systems of partial differential equations, and this graph is then related to the mesh. We propose the use of pr...
متن کاملAdaptive wavelet methods for saddle point problems
Recently, adaptive wavelet stratégies for symmetrie, positive definite operators have been introduced that were proven to converge. This paper is devoted to the generalization to saddle point problems which are also symmetrie, but indefinite. Firstly, we investigate a posteriori error estimâtes and generalize the known adaptive wavelet strategy to saddle point problems. The convergence of this ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: EURO journal on computational optimization
سال: 2022
ISSN: ['2192-4406', '2192-4414']
DOI: https://doi.org/10.1016/j.ejco.2022.100048